Method and apparatus for locating information from surroundings

ABSTRACT

An apparatus includes at least one a processor, and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to determine that an input defining a piece of information to be located has been received, determine the location of the information in an environment of the apparatus, and report the location.

RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.12/817,748 filed Jun. 17, 2010, the disclosure of which is incorporatedherein by reference in its entirety.

BACKGROUND

Locating an item in the surrounding physical environment may bedifficult, especially in a crowded situation. For example, it may bedifficult to locate a particular magazine in a rack with hundreds ofdifferent publications. It may also be challenging to locate a bookpositioned on a large shelf with many other titles in a library orbookstore. In a crowded city, it may be difficult for a foreigner tolocate text in an unfamiliar language that may be displayed on buildingsand street signs.

It would be advantageous to provide a facility for identifying desiredinformation within an environment where the information is not readilydiscernible, such as in a crowded setting.

SUMMARY

An apparatus is disclosed including at least one processor and at leastone memory including computer program code, the at least one memory andthe computer program code configured to, with the at least oneprocessor, cause the apparatus at least to determine that an inputdefining a piece of information to be located has been received,determine the location of the information in an environment of theapparatus, and report the location.

A method is disclosed including determining that an input defining apiece of information to be located has been received by an apparatus,determining the location of the information in a surrounding environmentof the apparatus, and reporting the location.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing aspects and other features of the presently disclosedembodiments are explained in the following description, taken inconnection with the accompanying drawings, wherein:

FIG. 1 illustrates a block diagram of an exemplary embodiment in theform of an apparatus;

FIG. 2 shows a flow diagram illustrating the operations of theapparatus;

FIG. 3 illustrates another exemplary embodiment in the form of acomputing device;

FIG. 4 shows a flow diagram illustrating exemplary operations of thecomputing device;

FIGS. 5A and 5B illustrate another exemplary embodiment in the form of amobile communications device;

FIG. 5C illustrates a block diagram of the mobile communications device;

FIG. 6 in combination with FIGS. 7A-7H illustrate exemplary operationsof the mobile communications device;

FIG. 8 shows an exemplary embodiment where the mobile communicationsdevice 500 is configured to communicate with a mobile telecommunicationsnetwork; and

FIG. 9 shows the operations of the mobile communications network inproviding the locating application as a service.

DETAILED DESCRIPTION

FIG. 1 illustrates a block diagram of an exemplary embodiment in theform of an apparatus 100. As a non-limiting example, the apparatus 100may be an integrated circuit. Apparatus 100 includes a processor 105 anda memory 110 including computer program code 115. The memory 110 and theprogram code 115 are configured to, with the processor 105, cause theapparatus 100 at least to determine a location of predefined informationin an environment 120 of the apparatus. The processor 105, and the otherprocessors disclosed in this application, may include one or more of amicroprocessor, a digital signal processor, a special purpose computerchip, a field programmable gate array, a controller an applicationspecific integrated circuit a computer or any suitable processor. Thememory 110, and the other memories disclosed in this application, mayinclude any non-transitory computer readable medium, for example, one ormore of a magnetic media, diskette, disk, computer hard drive, opticalmedia, compact disk, digital versatile disk, semiconductor, read-onlymemory, random access memory, or any suitable program code storagemedium. The memory 110 and the program code 115 may also be configuredto, with the processor 105, cause the apparatus 100 at least todetermine that an input defining a piece of information to be locatedhas been received, determine the location of the information in theenvironment 120 of the apparatus, and report the location. The programcode 115 may include an application 125 configured to, with theprocessor 105, cause the apparatus 100 to search for the information inthe environment 120, and upon locating the information, report thelocation for further use.

FIG. 2 shows a flow diagram illustrating this process. In block 200, theapparatus detects that a piece of information to be located has beenreceived. In block 205 the location of the information in theenvironment of the apparatus is determined, and in block 210, thelocation of the information is reported.

FIG. 3 illustrates another exemplary embodiment in the form of acomputing device 300. Apparatus 100 may be included as part of thisembodiment. Computing device 300 may be a personal computer, a tabletcomputer, touch pad device, Internet tablet, a laptop, desktop computeror any suitable computing device. Computing device 300 may be fixed orportable and typically includes a processor 305, a memory 310 withcomputer program code 315, and a user interface 320. The user interface320 may generally include user interface circuitry and the program code315 may include user interface program code 335 for controlling the userinterface 320. The user interface generally includes one or more inputdevices 325 and one or more output devices 330. The input and outputdevices may include one or more of a keyboard, a number of hard or softkeys, a display, a touch screen, or any other devices suitable forreceiving input and for providing output. The computing device 300 mayalso include a scanner 345 for surveying the environment in which thecomputing device is located. The scanner 345 may be included as part ofthe input devices 325 or output devices 330 and may comprise a camera orany other circuitry configured to collect information or characteristicsfrom the environment surrounding the computing device 300. The programcode 315 includes a locating application 340 for analyzing theinformation collected by the scanner 345 and recognizing predeterminedinformation or a specific pattern within the collected information.

FIG. 4 shows a flow diagram illustrating exemplary operations of thecomputing device 300. In block 400, a user provides input to thecomputing device 300 in the form of information to be located within thephysical surroundings. As a non-limiting example, if the information tobe located is a text string, the user may type the text using thekeyboard, touch screen, hard keys or other appropriate part of the inputdevices 325. If the information is graphical, for example, a logo, theuser may use the scanner to take a picture of an example of the logo.Alternately, the user may draw a depiction of the text, logo, or otherinformation on the touch screen if the computing device 300 is soequipped. In block 405, the locating application 340 enables the scanner345 and captures data from the environment. The scanner 345 may bepointed in a particular direction or toward an object to be surveyed.The locating application may also provide interactive directions to theuser with respect to pointing the scanner 345. In block 410, theapplication analyses the captured data for patterns that match orpartially match the input. In block 415, the matching or partiallymatching patterns are reported. For example, as the scanner 345 ispointed in a particular direction, the data collected by the scanner isprovided on a display. Matches or partial matches may be reported byhighlighting them on the display with different colors, outlines, orother distinguishing characteristics.

FIGS. 5A and 5B illustrate another exemplary embodiment in the form of amobile communications device 500. The computing device 300 may beincluded as a part of the mobile communication device 500. The mobilecommunications device 500 may be a mobile terminal, a cellular/mobilephone, a multimedia device, a personal communicator, a personal digitalassistant, or any other suitable device. Mobile communications device500 includes a user interface 540 with a display 505 and a keypad 510.The keypad 510 may include any suitable input devices such as, forexample, a multi-function/scroll key 515, soft keys 520, 525, andalphanumeric keys 530. Mobile communications device 500 also includes animage capture device 535 such as a camera as a further input device. Thedisplay 505 may include a touch screen and a graphical user interface.The mobile communications device 500 may also include a key 545 forenabling the image capture device 535. The user interface may alsoinclude a microphone 546 and a speaker 547 for inputting and outputtingspeech and oral commands.

FIG. 5C illustrates a block diagram of the mobile communications device500. As mentioned above, the device 500 includes a user interface 540with a display 505 and a keypad 510. The mobile communications device500 also includes one or more processors 548 and memories 550 includingcomputer program code 555 that further includes a locating application575, similar to locating application 340, for detecting a user inputdefining information to be located within the environment surroundingthe mobile communications device 500, scanning the environment anddetermining the location of the information, and providing the locationto the user. The mobile communications device 500 may also includecommunication circuitry 560 for exchanging information with otherdevices. The communication circuitry 560 generally includes a receiver565 and a transmitter 570 for network communication. The communicationcircuitry 560 is generally configured to allow the mobile communicationsdevice 500 to receive and send communications and messages, such asvoice transmissions, text messages, chat messages, multimedia messages,still images, video and email, over a network, such as a wired orwireless network.

FIG. 6 in combination with FIGS. 7A-7H illustrate exemplary operationsof this embodiment. In block 600, a user begins a search for an itemamong a large number of items 700 on the shelves of a grocery store orsupermarket, as shown in FIG. 7A. In block 605, the locating applicationis launched, by way of a key press or other operation. The mobilecommunications device 500 may provide an indication 705 that thelocation application 575 has been launched as shown in FIG. 7B. Afterlaunching, the mobile communication device 500 may also providedirections or guidance for further operations, by text, voice commands,or other indicators. In block 610, the mobile communication devicepresents a query for the search information, shown as item 710 in FIG.7C. In block 615, a user may respond to the query by providing an inputthat defines a piece of information to be located. As mentioned above,the information may include text, a picture from the scanner, a drawndepiction of the information, or any form of information that the mobilecommunication device 500 is capable of recognizing. In the example shownin FIG. 7D, a text input 715 is provided.

In block 620, the image capture device is aimed at a part of thephysical environment to be searched, in this example, the items 700 onthe shelves of the grocery store as shown in FIG. 7E. The portion of theenvironment may be provided on the display 505 to ensure that the properarea is being searched. In bock 625, the locating application may begincapturing and analyzing the displayed data automatically, or may beginthe capture and analysis in response to an input, for example, a keypress. As part of the analysis, the locating application 575 may use astatistical, syntactic, or other pattern recognition technique, asappropriate. The locating application 575 may include or have access toa database 580 of models to be used for comparison with the collecteddata to identify the matches with the search information. The modelscould include letters of selected alphabets in different typefaces foridentifying text. The models could also include known street names,commercial business names and logos, publication titles, book titles,names of different food and drink items and producers, and any othermodels that may be applicable. In the event that a picture or a drawndepiction is submitted as an input, the locating application cangenerate an additional model based on the submitted information.

In block 630, as an optional operation for efficiency, the locatingapplication may initially attempt to recognize certain predefined shapeswithin the collected data related to the information to be located. Forexample, if a logo is provided as the input, the locating applicationmay categorize the input as a combination of known geometrical shapesand then search for the combination. As another example, if text isprovided, the locating application may initially search for, identify,and highlight, shapes 720 that generally enclose text, for example,rectangles and squares as shown in FIG. 7F.

In block 635, in the example where the certain predefined shapes withinthe collected data are recognized, the location application 575 mayproceed to identify text 725 within the recognized predefined shapes, asshown in FIG. 7G. Otherwise, in block 640, the location application 575proceeds to identify items in the collected data that match or at leastpartially match the search information. In block 640, the matching data730 (and partial matches if applicable) may be displayed, as shown inFIG. 7H and the mobile communication device 500 may provide a visual oraudible alert. Once displayed, the user may now locate the item in thephysical environment 700.

FIG. 8 shows an exemplary embodiment where the mobile communicationsdevice 500 is configured to communicate with a mobile telecommunicationsnetwork 810 and the locating application 575 is provided as a service bythe mobile telecommunications network 810. In addition to the mobilecommunication device 500, other mobile terminals 800 may be connected tothe mobile telecommunications network 810 through radio frequency (RF)links 802, 808 via base stations 804, 809. The mobile telecommunicationsnetwork 810 may be in compliance with any commercially available mobiletelecommunications standard such as for example the global system formobile communications (GSM), universal mobile telecommunication system(UMTS), digital advanced mobile phone service (D-AMPS), code divisionmultiple access 2000 (CDMA2000), wideband code division multiple access(WCDMA), wireless local area network (WLAN), freedom of mobilemultimedia access (FOMA) and time division-synchronous code divisionmultiple access (TD-SCDMA).

The mobile telecommunications network 810 may be operatively connectedto a wide-area network 820, which may be the Internet or a part thereof.An Internet server 822 has data storage 824 and is connected to the widearea network 820. The server 822 may host a worldwide web/wirelessapplication protocol server capable of serving worldwide web/wirelessapplication protocol content to the mobile terminal 500. The mobiletelecommunications network 810 may include one or more mobiletelecommunications network servers 840. At least one of the mobiletelecommunications network servers 840 has a memory 845 and is alsoconnected to the mobile telecommunications network 810. A publicswitched telephone network (PSTN) 830 may be connected to the mobiletelecommunications network 810, and one or more telephone terminals 832may also be connected to the public switched telephone network 830. Themobile terminal 500 may also be configured for communicating to a device803 via a link 801, including, for example, one or more of a piconet,Bluetooth™, USB, IEEE 802.11, RS-232, or any other suitable link. Inthis embodiment, the locating application 575 may be located as computerprogram code in a non-transitory computer readable medium in the memory845.

FIG. 9 shows the operations of mobile communications network 810 inproviding the locating application 545 as a service. In block 900, arequest for initiation of the location application is generated at themobile telecommunications device 500. In block 905, the request iscommunicated to the mobile telecommunications network server 840 thoughthe mobile telecommunications network 810. In block 910, the applicationmay optionally be delivered or downloaded to the mobile device 500 andmay operate as described above with respect to the mobile communicationdevice embodiment. There may be a charge or fee for delivering theapplication. Alternately, as shown in block 915, the locatingapplication 575 may be run from the mobile telecommunications networkserver 840, where the memory 845 of the mobile telecommunicationsnetwork server operates as the memory 550 of the mobile communicationdevice 500 and provides the functions of the application 575 over themobile telecommunications network 810. As a further alternative, themobile communication device 500 and the mobile telecommunicationsnetwork server 840 may share portions of the locating application 575where the mobile communication device 500 performs some operations andthe mobile telecommunications network 810 performs other operations asappropriate given different operating conditions, for example, networktraffic and computing power available at the communication device 500and at the mobile telecommunications network server 840. There may be acharge or fee for providing the application in whole or in part over themobile telecommunications network 810.

The four embodiments disclosed herein provide for locating informationin the local environment in an expeditious manner, for example,especially when confronted with an overwhelming amount of information,information in another language, or when there is a need to locateinformation quickly.

As used in this application, the term ‘circuitry’ refers to all of thefollowing: (a) hardware-only circuit implementations (such asimplementations in only analog and/or digital circuitry) and (b) tocombinations of circuits and software (and/or firmware), such as (asapplicable): (i) to a combination of processor(s) or (ii) to portions ofprocessor(s)/software (including digital signal processor(s)), software,and memory(ies) that work together to cause an apparatus, such as amobile phone or server, to perform various functions) and (c) tocircuits, such as a microprocessor(s) or a portion of amicroprocessor(s), that require software or firmware for operation, evenif the software or firmware is not physically present.

This definition of ‘circuitry’ applies to all uses of this term in thisapplication, including in any claims. As a further example, as used inthis application, the term “circuitry” would also cover animplementation of merely a processor (or multiple processors) or portionof a processor and its (or their) accompanying software and/or firmware.The term “circuitry” would also cover, for example and if applicable tothe particular claim element, a baseband integrated circuit orapplications processor integrated circuit for a mobile phone or asimilar integrated circuit in server, a cellular network device, orother network device.”

It should be understood that the foregoing description is onlyillustrative of the present embodiments. Various alternatives andmodifications can be devised by those skilled in the art withoutdeparting from the embodiments disclosed herein. Accordingly, theembodiments are intended to embrace all such alternatives, modificationsand variances which fall within the scope of the appended claims.

What is claimed is:
 1. A method comprising: receiving a user input, atan apparatus, that identifies an object whose location is sought in aphysical environment surrounding the apparatus; determining, in responseto the user input, at least one identifying feature of the object based,at least in part, on the user input; capturing an image of the physicalenvironment; determining, in response to the user input and thecapturing of the image of the physical environment, that the object ispresent within the physical environment by way of analyzing, with aprocessor, the image of the physical environment for patterns that atleast partially match the identifying feature; and causing the objectwithin the image of the physical environment to be highlighted inresponse to the object being determined to be present within the imageof the physical environment.
 2. The method of claim 1, wherein the userinput includes the at least one identifying feature of the object. 3.The method of claim 1, wherein analyzing the image of the physicalenvironment for patterns that at least partially match the identifyingfeature comprises comparing the captured image to a database of models.4. The method of claim 3, wherein the user input comprises at least oneof a drawn depiction or an image of at least a portion of the object,and determining at least one identifying feature of the object comprisesgenerating at least one additional model from at least one of the drawndepiction or the image of at least a portion of the object.
 5. Themethod of claim 1, wherein highlighting the object comprises displayingthe image of the physical environment; and highlighting at least aportion of the image of the physical environment that at least partiallymatches the identifying feature.
 6. The method of claim 1, whereincapturing the image of the physical environment comprises providinginteractive directions for capturing the image of the physicalenvironment.
 7. A non-transitory computer-readable storage medium havingcomputer-executable program code instructions stored therein, thecomputer-executable program code instructions, when executed by aprocessor cause an apparatus to perform: receipt of a user input, at anapparatus, that identifies an object whose location is sought in aphysical environment surrounding the apparatus; determination, inresponse to the user input, of at least one identifying feature of theobject based, at least in part, on the user input; capture of an imageof the physical environment; determination, in response to the userinput and the capturing of the image of the physical environment, thatthe object is present within the physical environment by way ofanalysis, with a processor, of the image of the physical environment forpatterns that at least partially match the identifying feature; andcausation of the object within the image of the physical environment tobe highlighted in response to the object being determined to be presentwithin the image of the physical environment.
 8. The medium of claim 7,wherein the analysis of the image for patterns that at least partiallymatch the identifying feature comprises comparing the captured image toa database of models.
 9. The medium of claim 8, wherein the user inputcomprises at least one of a drawn depiction or image of at least aportion of the object, and the determination of at least one identifyingfeature of the object comprises generation of at least one additionalmodel from at least one of the drawn depiction or the image of at leasta portion of the object.
 10. The medium of claim 7, wherein thehighlighting of the object comprises displaying the image of thephysical environment and highlighting at least a portion of the image ofthe physical environment that at least partially matches the identifyingfeature.
 11. The medium of claim 7, wherein the capture of the image ofthe physical environment comprises providing of interactive directionsfor capturing the image of the physical environment.
 12. An apparatuscomprising: at least one processor; and at least one memory includingcomputer program code, the at least one memory and the computer programcode configured to, with the at least one processor, cause the apparatusat least to perform: receipt of a user input, at an apparatus, thatidentifies an object whose location is sought in a physical environmentsurrounding the apparatus; determination, in response to the user input,of at least one identifying feature of the object based, at least inpart, on the user input; capture of an image of the physicalenvironment; determination, in response to the user input and thecapturing of the image of the physical environment, that the object ispresent within the physical environment by way of analyzing, with aprocessor, the image of the physical environment for patterns that atleast partially match the identifying feature; and causation of theobject within the image of the physical environment to be highlighted inresponse to the object being determined to be present within the imageof the physical environment.
 13. The apparatus of claim 12, wherein theanalysis of the image for patterns that at least partially match theidentifying feature comprises comparison of the captured image to adatabase of models.
 14. The apparatus of claim 12, wherein the userinput comprises at least one of a drawn depiction or image of at least aportion of the object, and the determination of at least one identifyingfeature of the object comprises generation of at least one additionalmodel from at least one of the drawn depiction or the image of at leasta portion of the object.
 15. The apparatus of claim 12, wherein thehighlighting of the object comprises display of the image of thephysical environment and highlighting at least a portion of the image ofthe physical environment that at least partially matches the identifyingfeature.
 16. The apparatus of claim 12, wherein the capture of the imageof the physical environment comprises providing of interactivedirections for capturing the image of the physical environment.
 17. Theapparatus of claim 12, wherein the at least one memory and the computerprogram code are further configured to, with the at least one processor,cause the apparatus at least to determine that the user input comprisesat least one of text or graphics.
 18. The apparatus of claim 12, whereinthe analysis of the image of the physical environment for patterns thatat least partially match the identifying feature comprises analysis ofthe image for patterns that at least partially match the user input. 19.The apparatus of claim 12, wherein the highlighting comprises signifyingthe object by way of display of at least one distinguishingcharacteristic.
 20. The apparatus of claim 19, wherein thedistinguishing characteristic is at least one of a different color froma color of the object or an outline around the object.